Shanghai AI Lab, together with Ruijin Hospital affiliated to Shanghai Jiao Tong University School of Medicine, has released the medical multimodal foundational model group 'PuYi 2.0'. PuYi 2.0 adds multi-domain models and incremental language parameters, covering various data modalities such as medical images, medical texts, and bioinformatics. New open-source datasets include the medical image segmentation dataset SA-Med2D-20M and the pathology dataset SNOW. PuYi 2.0 incorporates an evaluation module to provide reference implementations for medical model capabilities, delivering a one-stop open-source solution for large medical models.

1. Pu Yu Ling Bi enables fluent bilingual (Chinese and English) interactions with images, accurately understanding image content and demonstrating a profound knowledge of Chinese culture. 2. Pu Yu Ling Bi unlocks a new creative ability for rich text and imagery articles, allowing users to generate articles with images by simply providing a topic. 3. Pu Yu Ling Bi employs a 'three-step' algorithmic process for creating image-text articles: generating text, planning illustrations, and intelligently selecting images. 4. In evaluations across several mainstream multimodal models, Pu Yu Ling Bi consistently performs at a leading level, particularly excelling in Chinese multimodal understanding. 5. Pu Yu

Shanghai AI Lab has released the large model training toolbox XTuner, which supports various hardware adaptations, significantly lowering the training cost barrier. XTuner is compatible with multiple open-source large models and can execute tasks such as incremental pre-training and instruction fine-tuning. XTuner balances usability and configurability with a built-in standardized process for one-click training.

Shanghai AI Lab, in collaboration with the Corpus Data Alliance, has open-sourced the 'Shusheng・Wanjuan' 1.0 multi-modal pre-trained dataset, which includes text, images, and video datasets, totaling over 2TB. The dataset has undergone fine-grained cleaning and deduplication, featuring multi-dimensional integration, meticulous processing, and ease of use. The release of this open-source dataset will help promote the application and innovation of large models and lower the technical barriers associated with large model technologies.